The Alpha 21364 network architecture

Advances in semiconductor technology have let microprocessors integrate more than a 100 million transistors on a single chip. The Alpha 21364 microprocessor uses 152 million transistors to integrate an Alpha 21264 processor core, a 1.75-Mbyte second-level cache, cache coherence hardware, two memory controllers, and a multiprocessor router on a single die, as Figure 1a shows. In the 0.18-micron bulk CMOS process, the 21364 will run at 1.2 GHz and provide 12.8 Gbytes/s of local memory bandwidth and 22.4 Gbytes/s of router bandwidth, as Figure 2 shows. The Alpha 21364’s tightly coupled multiprocessor network connects up to 128 such processors in a 2D torus network; Figure 1b shows a 12-processor configuration. A fully configured, 128-processor, shared-memory system can support up to 4 terabytes of Rambus memory and hundreds of terabytes of disk storage. We could also easily redesign the 21364 to support a much larger configuration. This multiprocessor configuration supports the massive computation and communication requirements of various application domains, such as high-performance technical computing, database servers, Web servers, and telecommunications. We designed the Alpha 21364 network architecture to meet the communication demands of these memoryand I/O-intensive applications. The novelty of the Alpha 21364’s router architecture lies in its extremely low latency, enormous bandwidth, and support for directory-based cache coherence. The router offers extremely low latency because it operates at 1.2 GHz, the same clock speed as the processor core. The pin-to-pin latency within the router is 13 cycles or 10.8 ns. In comparison, the ASIC-based SGI Spider router runs at 100 MHz and offers a 40-ns pin-topin latency. Similarly, the Alpha 21364 offers an enormous amount of peak and sustained bandwidth. The 21364 router can sustain between 70 and 90 percent of its 22.4-Gbytes/s peak bandwidth. The 21364’s router can offer such enormous bandwidth because of aggressive routing algorithms, carefully crafted distributed arbitration schemes, large amounts of on-chip buffering, and a fully pipelined router implementation. Finally, the network and router architectures have explicit support for directory-based cache coherence, such as separate virtual channels for different coherence protocol packet classes. This helps avoid deadlocks and improves the performance of the 21364’s coherence protocol. Shubhendu S. Mukherjee